Geolocation with aerial and satellite photography

ABSTRACT

A method of automatically geolocating a visual target. The method comprises operating a flying vehicle in a search region including the visual target. The method further includes affirmatively identifying a visual target in an aerial photograph of the search region captured by the flying vehicle. The method further includes automatically correlating the aerial photograph of the search region to a geo-tagged photograph of the search region, wherein the geo-tagged photograph is labelled with pre-defined geospatial coordinates. Based on such automatic correlation, a geospatial coordinate is determined for the visual target in the search region.

BACKGROUND

Portable computing devices may use global positioning system (GPS)technology to determine a geolocation (e.g., of a user or of a nearbylandmark). However, in some examples, reliable GPS information may beunavailable. For example, GPS information may be incomplete and/orerroneous due to compromised communication with one or more GPSsatellite signals, or as a result of spoofing or jamming.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

A method is disclosed of automatically geolocating a visual target,including operating a flying vehicle in a search region including avisual target. The method further includes affirmatively identifying avisual target in an aerial photograph of the search region captured bythe flying vehicle. The method further includes automaticallycorrelating the aerial photograph of the search region to a geo-taggedphotograph of the search region, wherein the geo-tagged photograph islabelled with pre-defined geospatial coordinates. Based on suchautomatic correlation, a geospatial coordinate is determined for thevisual target in the search region.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows a geolocation system.

FIG. 1B shows exemplary aerial and satellite photography.

FIG. 2 shows a method of automatically geolocating a visual target.

FIG. 3 shows an exemplary computing system.

DETAILED DESCRIPTION

Geolocation services may enable humans to make more informed decisions,for example regarding navigation. “Geolocation” may be used herein todescribe any relevant information pertaining to a spatial location onEarth (e.g., as latitude/longitude coordinates, as an address, and/orany other suitable indicator of a location). For example, a computerdevice configured to provide geolocation services may enable a user ofthe computer device to more efficiently navigate to an address,landmark, natural feature or other destination/location. For example, auser's current geolocation and the geolocation of a planned destinationmay be used to automatically determine navigational information (e.g.,direction and/or distance, and/or navigational steps from the currentgeolocation to the planned destination).

Global positioning system (GPS) devices are frequently employed toprovide geolocation services. GPS devices interpret a plurality ofsignals from GPS satellites, in order to triangulate a real-worldlocation based on the transmission latencies of the GPS signals. When aGPS device is able to receive sufficient, accurate GPS signals from aplurality of GPS satellites, it may be able to accurately determine ageolocation of the GPS device.

However, in some examples, reliable GPS signals may be not be available.For example, the GPS device may be unable to receive signal from some orall of the satellites (e.g., due to a satellite failing to broadcast GPSsignal, and/or due to environmental attenuation of broadcast signals,such as occlusion by steep rocky terrain, buildings, etc.). As anotherexample, the GPS device may receive spurious signals. For example, insome cases the GPS signal bandwidth may include unauthorized signalsand/or noise, decreasing signal-to-noise ratio for true GPS signals. Insome examples, a malicious party may broadcast fake GPS signals in theGPS signal bandwidth. In general, falsified GPS signals may or may notbe detectable. In some examples, a falsified GPS signal may correspondto a location which can be determined to be inaccurate (e.g., thefalsified GPS signal may correspond to a location that is miles awayfrom user's location). However, in some examples a malicious party maybroadcast GPS signals that result in undetectable, yet significanterrors to GPS determination of geolocation. For example, the maliciousparty may broadcast a falsified GPS signal that is gradually alteredover time (e.g., skewing a reported clock time in the falsified signalgradually over time). Such malicious techniques may result inaccumulated errors to an apparent GPS-determined geolocation, while notpresenting any obvious evidence of falsification upon initial receptionof the falsified GPS signal.

Accordingly, the present disclosure is directed to a geolocationtechnique that may be used when GPS signals may be unavailable and/orunreliable. Using the methodology of the present disclosure, visualimagery of a target location (e.g., photographs) is correlated togeo-tagged imagery (e.g., geo-tagged satellite photographs or any othergeo-tagged photographs) that is labeled with geospatial coordinates(e.g., labeled photographs, with geospatial coordinates associated withX-Y pixel coordinates of the photographs). Based on such correlation, ageolocation of the target location is determined (e.g., geospatialcoordinates of the target location may be determined based on X-Ycoordinates of pixels in the correlated photos). The geolocationtechniques of the present disclosure may be used to verifyGPS-determined geolocation and/or to correct deficiencies inGPS-determined geolocation. For example, the techniques of the presentdisclosure may allow a user to obtain an accurate geolocation (e.g., ofthe user and/or of a nearby landmark), even when GPS signals areunavailable and/or compromised by a malicious party. The terms “imagery”and “photography” may be used interchangeably herein to describe anysuitable representation of visible features in a real-world environment.As a non-limiting example, photography may be captured using an opticalsystem, e.g., based on exposure of a sensor medium to light reflectedoff of the features in the real-world environment.

FIG. 1A shows a geolocation system 100. Geolocation system 100 includesan assistive device 102, which in typical examples is a portable devicecarried by or along with a human user. As a non-limiting example,assistive device 102 may be a head-mounted device worn by a human user.As further non-limiting examples, assistive device 102 may be a mobilehandheld device (e.g., a smartphone), a laptop device, an automatedassistant speaker interface device (e.g., a smart speaker), or a vehiclemultimedia interface device (e.g., a car heads-up display).

In some examples, assistive device 102 is configured to determinegeospatial coordinates (e.g., of the user and/or of a nearby object,person, or landmark) and to present determined geospatial coordinates tothe user. As a non-limiting example, the determined geospatialcoordinates may be visually presented via a display (e.g., via ahead-mounted display of a head-mounted device). In some examples,instead of or in addition to visual presentation, the determinedgeolocation may be presented via text, presented via audio (e.g., via aheadset of the assistive device 102), presented via a virtual realityand/or augmented reality interface, and/or presented in any othersuitable fashion (e.g., as an interactive and/or non-interactive map, aspart of a series of navigation directions, as a geospatial coordinate,and/or as an address). In other examples, the methodology of the presentdisclosure may be utilized to locate a visual target (e.g., a user) evenwhen the visual target is not carrying an assistive device 102. As anon-limiting example, the above-described steps of determininggeospatial coordinates and visually presenting geospatial coordinatesmay be carried out at remotely at a computer device associated with asupport team of a user, in order to locate the user and report thedetermined location to the support team.

According to the techniques of the present disclosure, assistive device102 or any other computer device may be configured to determine thegeolocation via one or more means including correlation of photographyof a surrounding environment (e.g., aerial photography or any othersuitable photography) to geo-tagged photography (e.g., satelliteimagery, and/or other pre-recorded aerial imagery). The present exampleincludes numerous examples of geolocation using correlation of aerialphotography to satellite photography. However, the techniques describedherein may be implemented by comparing any other photography of thesurrounding environment to geo-tagged photography labelled with ageo-referenced position (e.g., pre-recorded satellite imagery labelledwith geospatial coordinates). In general, as will be described furtherbelow with regard to method 200, a target is affirmatively identified inphotography of the environment (e.g., in aerial photography). The targetmay be identified by human evaluation and/or automatically identified bymachine learning. As described herein, identification of the target inthe environment photography may include determining a location of thetarget in the environment photography (e.g., an X-Y pixel coordinate inan aerial photograph). Accordingly, visual features of the environmentphotography may be correlated to a geo-tagged photograph (e.g.,satellite photograph) and the location of the target in the photographmay be correlated to a geospatial coordinate based on the labels of thegeo-tagged photograph.

Assistive device 102 may optionally also be configured to measure thegeolocation via GPS. Accordingly, assistive device 102 may optionallyinclude a GPS configured to estimate a geolocation via received GPSsignals. As described above, geolocation estimated via the GPS may ormay not be accurate. Therefore, assistive device 102 is configured tooptionally corroborate received GPS signals with geolocation determinedvia correlation of environmental imagery (e.g., aerial photography) tosatellite (or other pre-recorded and/or geo-tagged) photography.

Turning briefly to FIG. 1B, FIG. 1B includes an exemplary image of anenvironment 110, for example an aerial photograph of an environmentincluding a building 110 a, a human 110 b, and a tree 110 c. Forexample, human 110 b may be a human user of an assistive device 102using the assistive device 102 to request geospatial coordinates oftheir current location. FIG. 1B further includes an exemplary satellitephotograph 112 showing an overhead satellite view of the building 112 a,overhead view of the tree 112 c, and an overhead view of additionaltrees 112 d on the other side of the building. Using the methods of thepresent disclosure, the aerial photograph of the environment 110 may becorrelated to the satellite photograph 112 to determine a location 112 bcorresponding to the human 110 b. Based on the determined location 112 bin the satellite photography, a geospatial coordinate of the user 110 bmay be determined and returned to the assistive device. Furthermore,using the methods of the present disclosure, the assistive device mayadditionally or alternately obtain locations for the tree 112 c orbuilding 112 a, for example.

Returning to FIG. 1A, as described above, assistive device 102 isconfigured to determine a geospatial coordinate in the nearbyenvironment using photography of the nearby environment (e.g., one ormore aerial photograph(s)) and geo-tagged (e.g., satellite) photographylabeled with geospatial coordinates. The geo-tagged photography may belabeled in any suitable fashion. For example, a satellite photo may belabeled with one or more coordinates associated with specific pixels(e.g., a coordinate associated with a corner or center of the image).Alternately or additionally, specific pixel locations in an image mayhave an associated label (e.g., indicating a geospatial coordinate,address, or landmark). The geospatial coordinate may be for any suitablelocation on the labeled, geo-tagged photography. For example, ageospatial coordinate may be obtained for a position on the labeled mapthat corresponds to a current position of a human user of assistivedevice 102, a nearby human in the environment, a nearby object, alandmark, a vegetation feature, or any other visually-identifiablefeature in the environment . . . . For example, human user(s) ofassistive device 102 may request geolocation services to obtain ageospatial coordinate of themselves. Alternately or additionally, ahuman user may request geospatial coordinates of another nearby human,of a building, structure, or any other man-made object, or of ageographic landmark. Alternately or additionally, a different human(e.g., a member of a support team associated with the human user) mayrequest geolocation service of the human user on the human user'sbehalf.

In some examples, assistive device 102 may be configured to process oneor more photograph(s) of the nearby environment and one or moresatellite photograph(s) labeled with geospatial coordinates, in order todetermine the geospatial coordinate of the visual target, for example bycalculating an image correlation between the photographs using one ormore on-board processor(s) and/or storage device(s). Accordingly, insome examples, the labelled satellite photograph(s) may be pre-loaded toassistive device 102 for use in such processing. For example, assistivedevice 102 may be pre-configured with global and/or regional satellitedata with geospatial labels at any suitable resolution. For example,assistive device 102 may be configured to download a regional satelliteimage database including a plurality of satellite photographs andpre-defined geospatial coordinates, so that photography of the nearbyenvironment may be correlated against one or more of the plurality ofsatellite photographs (e.g., using any suitable image search algorithm,machine learning, image homography, random sample consensus (RANSAC),and/or any other suitable method as will be described further below withregard to method 200).

Alternately or additionally, assistive device 102 may be configured toco-operate with one or more external devices to obtain and/or processthe photography. Accordingly, assistive device 102 optionally includes acommunication subsystem configured to communicate with one or more otherdevices to obtain and/or process photography of the nearby environmentand/or labelled satellite photography. Assistive device 102 maycommunicatively couple to other devices via any suitable communicationtechnology (e.g., via Bluetooth, near-field communication, wireless meshnetwork, radio, wireless and/or wired local-area network connection). Insome examples, assistive device 102 may communicate with an externalserver 106. External server 106 may be any suitable computing device,e.g., a cloud computing server or an edge computing server. Edgecomputing may be used herein to refer to any device configured to supplyassistive device 102 with networking, processing, storage, and/or otherfunctionality, for example via a radio connection, wireless meshnetwork, local area network, etc.

In some examples, assistive device 102 may be configured to obtainphotography of the nearby environment with an on-board camera (e.g., acamera of an HMD device or mobile device, or a vehicular camera in anassistive system of an automobile).

Alternately or additionally, assistive device 102 may communicativelycouple to a different device configured to obtain photography of thenearby environment with a camera. For example, assistive device 102 maycommunicate with a flying vehicle 104 including a camera, wherein flyingvehicle 104 is configured to fly above a visual target to obtain anaerial picture of the visual target, and communicate the aerial pictureof the visual target back to assistive device 102. Assistive device 102may communicatively couple to the flying vehicle 104 via a radioconnection or any other suitable connection. In some examples, assistivedevice 102 may communicatively couple to the flying vehicle 104 via anintermediate server (e.g., assistive device 102 and flying vehicle 104may exchange messages via an external server 106).

As a non-limiting example, flying vehicle 104 may be a human-portableflying vehicle that may be deployed by a human user of assistive device102. Alternately or additionally, flying vehicle 104 may be aremote-controlled flying vehicle that may communicatively couple toassistive device 102 in order to enable the user to pilot theremote-controlled flying vehicle to obtain a suitable photograph.Alternately or additionally, flying vehicle 104 may be piloted at leastin part via commands issued by another human user (e.g., a remote pilotin a support team associate with the human user). In other examples,flying vehicle 104 may be not be human-portable and/or flying vehicle104 may not be deployed or piloted by the human user of assistive device102. Instead, flying vehicle 104 may be an autonomous, remotely piloted,and/or human-piloted flying vehicle deployed from any suitable location.For example, flying vehicle 104 may be a helicopter, plane, quadcopter,or any other state-of-the-art and/or future flying craft.

In general, the flying vehicle may be configured to deploy to an aerialregion substantially above the ground target in order to capture asuitable aerial photograph. For example, flying vehicle 104 may be ahuman-portable, autonomous flying vehicle configured to ascend to asuitable altitude to take a plan-view photograph of a nearby environmentof the human user, then to descend and return to the human user. Theaerial region substantially above the target may be at any locationsuitable for obtaining a photograph that is substantially in plan viewso that a visual target in the environment may be identified withoutobstruction. In some examples, perspective distortion to the image maybe at most a predefined threshold, e.g., at most 30 degrees, at most 15degrees, at most 5 degrees or at most 2 degrees, for example relative toa photo taken from immediately above the visual target and/or a phototaken at a substantial difference (e.g., a satellite photo).

Flying vehicle 104 may be configured to capture any suitable aerialphotography that may be correlated against geo-labelled satellitephotography. For example, flying vehicle 104 may be configured tocapture near-infrared photographs, suitable for assessing near-infraredfeatures for comparison to near-infrared features present innear-infrared satellite photography. Alternately or additionally, flyingvehicle 104 may be configured to capture color photography (e.g., forcorrelation to color satellite photography). Alternately oradditionally, flying vehicle 104 may be configured to capturehyperspectral photographs based on light in any two or more spectralbands (e.g., visible light and near-infrared, and/or any other spectralbands). Accordingly, such hyperspectral photographs may be correlatedagainst corresponding hyperspectral features in satellite photography.

The above-described methodology may be applied using any suitablecomputer device(s) and/or environmental photography, e.g., using computecapabilities of a suitable assistive device and aerial photographycaptured by a flying vehicle. Accordingly, FIG. 2 shows a method 200 ofautomatically geolocating a visual target. At 202, method 200 includesoperating a flying vehicle in a search region including a visual target.

At 204, method 200 includes affirmatively identifying a visual target inan aerial photograph of the search region captured by the flyingvehicle. In some examples, identifying the visual target in the aerialphotograph may be based on computer-identification using visual featuresin the photograph. For example, the visual target may be automaticallycomputer-identified using machine learning (e.g., using a convolutionalneural network or any other suitable algorithm). In other examples, thevisual target may be identified by a human. For example, the visualtarget may be identified by a human user of an assistive device or by amember of a support team of the human user (e.g., by pointing at alocation in an image captured by the flying vehicle). In general, theaffirmative identification of the visual target in the aerial photographmay include determining a location of the visual target in the aerialphotograph (e.g., as an X-Y pixel coordinate in the aerial photograph).For example, a machine learning system may be trained for objectdetection and/or object location, thereby permitting the machinelearning system to affirmatively identify a visual target in aphotograph, including determining a location of the visual target in thephotograph. Alternately or additionally, as described above, a humanuser may affirmatively identify the visual target in the aerialphotography by pointing at an aerial photograph, thereby indicating adetermined location of the visual target in the aerial photograph.

At 206, method 200 includes automatically correlating the aerialphotograph of the search region to a geo-tagged photograph of the searchregion. As a non-limiting example, the geo-tagged photograph may be asatellite photograph. The satellite photograph is labelled withpre-defined geospatial coordinates which may be determined in advanceand stored along with the satellite photograph—e.g., latitude/longitudevalues. The satellite photograph and/or labels may be stored on theassistive device (e.g., whether the assistive device is a head-mounteddevice, mobile device, or any other device), stored on the flyingvehicle, and/or stored on an external server. For example, the labelsmay be stored as part of the satellite photograph (e.g., encoded in animage file format used to store the satellite photograph) and/or storedseparately from the satellite photograph (e.g., stored in a separatedatabase, wherein the separate database is maintained on the samestorage device as the satellite photograph or on a different storagedevice). Accordingly, the automatic correlation may be performed on anysuitable device. For example, some or all of the automatic correlationmay be offloaded to an external server, in order to return results tothe assistive device. As a non-limiting example, automaticallycorrelating the aerial photograph of the search region to the satellitephotograph of the search region may include 1) communicating the aerialimage to a remote server, 2) computing the automatic correlation at theremote server, and 3) returning the computed automatic correlation tothe assistive device. In other examples, automatically correlating theaerial photograph of the search region to the satellite photograph ofthe search region includes 1) communicating the aerial image to theassistive device and 2) computing the automatic correlation at theassistive device.

Regarding transmission of aerial photography, it may be transmitted(e.g., between the flying vehicle, assistive device, and/or externalserver) in the form of raw images, tonemapped images (e.g. 8 bit perpixel data), compressed images (e.g. JPEG compressed), and/or high-levelimage features (e.g., compressed images and/or neural network featuresderived from images). Transmitting high-level image features may permitcorrelation of a photograph of the environment against satellitephotography with a reduced network bandwidth for transmission of theenvironment and/or satellite photography.

In some examples, automatically correlating the aerial photograph of thesearch region to the satellite photograph of the search region includesautomatically computer-assessing a homography between the aerialphotograph and the satellite photograph. In some examples, thehomography may be established via random sample consensus (RANSAC) orvia any other suitable statistical and/or machine learning algorithm.For example, RANSAC may be able to align geographic landmarks based onimage features. Furthermore, RANSAC may be robust against partialocclusion such as clouds and/or vegetation cover, so that a visualtarget in an aerial image may be correlated against a satellite imageeven when partially occluded in one or more of the aerial image and thesatellite image. In some examples, computer-assessing the homography isbased on comparing a set of derived image features of the aerialphotograph having a smaller dimensionality than the aerial photograph,to a corresponding set of derived image features of the satellitephotograph having a smaller dimensionality than the satellitephotograph. For example, a dimensionality of a raw image may depend on aresolution and color depth of the image, e.g., one dimension for eachpixel in the image having values depending on the color depth of theimage. Accordingly, the computer-assessed homography may be based oncompressed image features, features assessed by a neural network,frequency domain features (e.g., wavelet or FFT features in the spatialfrequency domain), and/or any other suitably reduced set of featureshaving a smaller dimensionality than the full pixel resolution of theimage.

At 208, method 200 includes, based on the above-described automaticcorrelation, determining a geospatial coordinate for the visual targetin the search region. For example, the RANSAC homography computation maybe configured to match the aerial photograph to the satellite photographand to use a scale and displacement in the matching of the photographsto determine a coordinate of a pixel location in the aerial photography.

In some examples, the above-described methodology may be used to correcterrors in a GPS system. For example, when the assistive device includesa GPS, the determined geospatial coordinate may be compared to ameasured location assessed by the GPS system, for example to assessaccuracy of the measured location assessed by the GPS system. Forexample, an error in the measured location assessed by the GPS systemmay be detected based on a disparity between (1) the geospatialcoordinate determined via the described image correlation and (2) themeasured location assessed by the GPS system. In some examples, anassistive device may be configured to primarily utilize GPS forgeolocation, while periodically checking the GPS-determined coordinatesagainst a determined geospatial coordinate from photography of theenvironment. For example, aerial photography may be used to periodicallydetermine that GPS measurements remain accurate and uncompromised.Furthermore, geospatial coordinates determined from photography of theenvironment may be used to assess a corrective model for the measuredlocation assessed by the GPS system based on the disparity. For example,if GPS signals in an area are compromised (e.g., falsified signals withGPS clock skew), the aerial photography may be used to develop acorrective clock model that accounts for skew in some or all of thecompromised signals, thereby improving an accuracy of GPS measurementsdespite the compromised GPS signals. In some examples, geospatialcoordinates determined from photography may be supplied, along withmeasured GPS coordinates, to a machine learning model configured tolearn a correlation between the geospatial coordinates determined fromphotography and the measured GPS coordinates. For example, based on suchcorrelation, the machine learning model may be operated to predictwhether measured GPS coordinates may be inaccurate (e.g., due tospoofing) and/or to correct inaccurate GPS coordinates by predictinglikely geospatial coordinates based on the model.

In some embodiments, the methods and processes described herein may betied to a computing system of one or more computing devices. Inparticular, such methods and processes may be implemented as acomputer-application program or service, an application-programminginterface (API), a library, and/or other computer-program product.

FIG. 3 schematically shows a non-limiting embodiment of a computingsystem 300 that can enact one or more of the methods and processesdescribed above. Computing system 300 is shown in simplified form.Computing system 300 may take the form of one or more personalcomputers, server computers, tablet computers, home-entertainmentcomputers, network computing devices, gaming devices, mobile computingdevices, mobile communication devices (e.g., smart phone), and/or othercomputing devices. For example, computing system 300 may be an assistivedevice 102.

Computing system 300 includes a logic machine 302 and a storage machine304. Computing system 300 may optionally include a display subsystem306, input subsystem 308, communication subsystem 310, and/or othercomponents not shown in FIG. 3 .

Logic machine 302 includes one or more physical devices configured toexecute instructions. For example, the logic machine may be configuredto execute instructions that are part of one or more applications,services, programs, routines, libraries, objects, components, datastructures, or other logical constructs. Such instructions may beimplemented to perform a task, implement a data type, transform thestate of one or more components, achieve a technical effect, orotherwise arrive at a desired result.

The logic machine may include one or more processors configured toexecute software instructions. Additionally or alternatively, the logicmachine may include one or more hardware or firmware logic machinesconfigured to execute hardware or firmware instructions. Processors ofthe logic machine may be single-core or multi-core, and the instructionsexecuted thereon may be configured for sequential, parallel, and/ordistributed processing. Individual components of the logic machineoptionally may be distributed among two or more separate devices, whichmay be remotely located and/or configured for coordinated processing.Aspects of the logic machine may be virtualized and executed by remotelyaccessible, networked computing devices configured in a cloud-computingconfiguration.

Storage machine 304 includes one or more physical devices configured tohold instructions executable by the logic machine to implement themethods and processes described herein. When such methods and processesare implemented, the state of storage machine 304 may betransformed—e.g., to hold different data.

Storage machine 304 may include removable and/or built-in devices.Storage machine 304 may include optical memory (e.g., CD, DVD, HD-DVD,Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM,etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive,tape drive, MRAM, etc.), among others. Storage machine 304 may includevolatile, nonvolatile, dynamic, static, read/write, read-only,random-access, sequential-access, location-addressable,file-addressable, and/or content-addressable devices.

It will be appreciated that storage machine 304 includes one or morephysical devices. However, aspects of the instructions described hereinalternatively may be propagated by a communication medium (e.g., anelectromagnetic signal, an optical signal, etc.) that is not held by aphysical device for a finite duration.

Aspects of logic machine 302 and storage machine 304 may be integratedtogether into one or more hardware-logic components. Such hardware-logiccomponents may include field-programmable gate arrays (FPGAs), program-and application-specific integrated circuits (PASIC/ASICs), program- andapplication-specific standard products (PSSP/ASSPs), system-on-a-chip(SOC), and complex programmable logic devices (CPLDs), for example.

The terms “module,” “program,” and “engine” may be used to describe anaspect of computing system 300 implemented to perform a particularfunction. In some cases, a module, program, or engine may beinstantiated via logic machine 302 executing instructions held bystorage machine 304. It will be understood that different modules,programs, and/or engines may be instantiated from the same application,service, code block, object, library, routine, API, function, etc.Likewise, the same module, program, and/or engine may be instantiated bydifferent applications, services, code blocks, objects, routines, APIs,functions, etc. The terms “module,” “program,” and “engine” mayencompass individual or groups of executable files, data files,libraries, drivers, scripts, database records, etc.

It will be appreciated that a “service”, as used herein, is anapplication program executable across multiple user sessions. A servicemay be available to one or more system components, programs, and/orother services. In some implementations, a service may run on one ormore server-computing devices.

The techniques of the present disclosure may be implemented using anysuitable combination of state-of-the-art and/or future machine learning(ML), artificial intelligence (AI), and/or natural language processing(NLP) techniques. Non-limiting examples of techniques that may beincorporated in an implementation of one or more machines includesupport vector machines, multi-layer neural networks, convolutionalneural networks (e.g., including spatial convolutional networks forprocessing images and/or videos, temporal convolutional neural networksfor processing audio signals and/or natural language sentences, and/orany other suitable convolutional neural networks configured to convolveand pool features across one or more temporal and/or spatialdimensions), recurrent neural networks (e.g., long short-term memorynetworks), associative memories (e.g., lookup tables, hash tables, BloomFilters, Neural Turing Machine and/or Neural Random Access Memory), wordembedding models (e.g., GloVe or Word2Vec), unsupervised spatial and/orclustering methods (e.g., nearest neighbor algorithms, topological dataanalysis, and/or k-means clustering), graphical models (e.g., (hidden)Markov models, Markov random fields, (hidden) conditional random fields,and/or AI knowledge bases), and/or natural language processingtechniques (e.g., tokenization, stemming, constituency and/or dependencyparsing, and/or intent recognition, segmental models, and/orsuper-segmental models (e.g., hidden dynamic models)).

In some examples, the methods and processes described herein may beimplemented using one or more differentiable functions, wherein agradient of the differentiable functions may be calculated and/orestimated with regard to inputs and/or outputs of the differentiablefunctions (e.g., with regard to training data, and/or with regard to anobjective function). Such methods and processes may be at leastpartially determined by a set of trainable parameters. Accordingly, thetrainable parameters for a particular method or process may be adjustedthrough any suitable training procedure, in order to continually improvefunctioning of the method or process.

Non-limiting examples of training procedures for adjusting trainableparameters include supervised training (e.g., using gradient descent orany other suitable optimization method), zero-shot, few-shot,unsupervised learning methods (e.g., classification based on classesderived from unsupervised clustering methods), reinforcement learning(e.g., deep Q learning based on feedback) and/or generative adversarialneural network training methods, belief propagation, RANSAC (randomsample consensus), contextual bandit methods, maximum likelihoodmethods, and/or expectation maximization. In some examples, a pluralityof methods, processes, and/or components of systems described herein maybe trained simultaneously with regard to an objective function measuringperformance of collective functioning of the plurality of components(e.g., with regard to reinforcement feedback and/or with regard tolabelled training data). Simultaneously training the plurality ofmethods, processes, and/or components may improve such collectivefunctioning. In some examples, one or more methods, processes, and/orcomponents may be trained independently of other components (e.g.,offline training on historical data).

When included, display subsystem 306 may be used to present a visualrepresentation of data held by storage machine 304. This visualrepresentation may take the form of a graphical user interface (GUI). Asthe herein described methods and processes change the data held by thestorage machine, and thus transform the state of the storage machine,the state of display subsystem 306 may likewise be transformed tovisually represent changes in the underlying data. Display subsystem 306may include one or more display devices utilizing virtually any type oftechnology. Such display devices may be combined with logic machine 302and/or storage machine 304 in a shared enclosure, or such displaydevices may be peripheral display devices.

When included, input subsystem 308 may comprise or interface with one ormore user-input devices such as a keyboard, mouse, touch screen, or gamecontroller. In some embodiments, the input subsystem may comprise orinterface with selected natural user input (NUI) componentry. Suchcomponentry may be integrated or peripheral, and the transduction and/orprocessing of input actions may be handled on- or off-board. Example NUIcomponentry may include a microphone for speech and/or voicerecognition; an infrared, color, stereoscopic, and/or depth camera formachine vision and/or gesture recognition; a head tracker, eye tracker,accelerometer, and/or gyroscope for motion detection and/or intentrecognition; as well as electric-field sensing componentry for assessingbrain activity.

When included, communication subsystem 310 may be configured tocommunicatively couple computing system 300 with one or more othercomputing devices. Communication subsystem 310 may include wired and/orwireless communication devices compatible with one or more differentcommunication protocols. As non-limiting examples, the communicationsubsystem may be configured for communication via a wireless telephonenetwork, or a wired or wireless local- or wide-area network. In someembodiments, the communication subsystem may allow computing system 300to send and/or receive messages to and/or from other devices via anetwork such as the Internet.

In an example, a method of automatically geolocating a visual targetcomprises: operating a flying vehicle in a search region including avisual target; affirmatively identifying the visual target in an aerialphotograph of the search region captured by the flying vehicle;automatically correlating the aerial photograph of the search region toa geo-tagged photograph of the search region, wherein the geo-taggedphotograph is labelled with pre-defined geospatial coordinates; andbased on such automatic correlation, determining a geospatial coordinatefor the visual target in the search region. In this or any otherexample, the geo-tagged photograph of the search region is a satellitephotograph of the search region. In this or any other example, thevisual target is a human user electronically requesting geolocation. Inthis or any other example, the human user carries an assistive deviceconfigured to present the determined geospatial coordinate. In this orany other example, the assistive device is a head-mounted device worn bythe human user. In this or any other example, the assistive deviceincludes a global positioning (GPS) system, and wherein the methodfurther comprises comparing the determined geospatial coordinate to ameasured location assessed by the GPS system. In this or any otherexample, the method further comprises detecting an error in the measuredlocation assessed by the GPS system based on a disparity between thedetermined geospatial coordinate and the measured location assessed bythe GPS system. In this or any other example, the method furthercomprises assessing a corrective model for the measured locationassessed by the GPS system based on the disparity. In this or any otherexample, automatically correlating the aerial photograph of the searchregion to the geo-tagged photograph of the search region includescommunicating the aerial image to a remote server, computing theautomatic correlation at the remote server, and returning the computedautomatic correlation to the assistive device. In this or any otherexample, automatically correlating the aerial photograph of the searchregion to the geo-tagged photograph of the search region includescommunicating the aerial image to the assistive device and computing theautomatic correlation at the assistive device. In this or any otherexample, the assistive device is configured to download a regional imagedatabase including the geo-tagged photograph and pre-defined geospatialcoordinates. In this or any other example, the flying vehicle is ahuman-portable device deployed by the human user. In this or any otherexample, the visual target is a geographic landmark. In this or anyother example, the visual target is a building, structure, or otherman-made object. In this or any other example, the flying vehicle is anautonomous flying vehicle configured to automatically deploy to anaerial region substantially above the visual target for capturing theaerial photograph. In this or any other example, the flying vehicle isconfigured to capture near-infrared photographs and wherein the aerialphotograph and the geo-tagged photograph include near-infrared features.In this or any other example, automatically correlating the aerialphotograph of the search region to the geo-tagged photograph of thesearch region includes automatically computer-assessing a homographybetween the aerial photograph and the geo-tagged photograph. In this orany other example, computer-assessing the homography is based on a setof derived image features of the aerial photograph having a smallerdimensionality than the aerial photograph, to a corresponding set ofderived image features of the geo-tagged photograph having a smallerdimensionality than the geo-tagged photograph.

In an example, a head-mounted device comprises: a global positioningsystem (GPS); a display; a logic device; and a storage device holdinginstructions executable by the processor to: receive an aerialphotograph of a search region including a visual target, the aerialphotograph captured by a flying vehicle operating in the search region;automatically correlate the aerial photograph of the search region to asatellite photograph of the search region, wherein the satellitephotograph is labelled with pre-defined geospatial coordinates; based onsuch automatic correlation, determine a geospatial coordinate for thevisual target in the search region; and detect an error in a measuredlocation assessed by the GPS system based on a disparity between thedetermined geospatial coordinate and the measured location assessed bythe GPS system.

In an example, a computer system comprises: a logic device; and astorage device holding instructions executable by the logic device to:receive an aerial photograph of a search region including a visualtarget, the aerial photograph captured by a flying vehicle operating inthe search region; automatically correlate the aerial photograph of thesearch region to a satellite photograph of the search region, whereinthe satellite photograph is labelled with pre-defined geospatialcoordinates; and based on such automatic correlation, determine ageospatial coordinate for the visual target in the search region.

It will be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated and/ordescribed may be performed in the sequence illustrated and/or described,in other sequences, in parallel, or omitted. Likewise, the order of theabove-described processes may be changed.

The subject matter of the present disclosure includes all novel andnon-obvious combinations and sub-combinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

The invention claimed is:
 1. A method of automatically geolocating avisual target, comprising: affirmatively identifying a visual target inan aerial photograph of a search region, the visual target having aglobal positioning system (GPS) device; automatically correlating theaerial photograph of the search region to a geo-tagged photograph of thesearch region, wherein the geo-tagged photograph is labelled withpre-defined geospatial coordinates; based on such automatic correlation,determining a geospatial coordinate for the visual target in the searchregion; receiving a measured location assessed by the GPS device;detecting an error in the measured location assessed by the GPS devicebased on a disparity between the determined geospatial coordinate andthe measured location assessed by the GPS device; and assessing acorrective model for the measured location assessed by the GPS devicebased on the disparity.
 2. The method of claim 1, wherein the geo-taggedphotograph of the search region is a satellite photograph of the searchregion.
 3. The method of claim 1, wherein the visual target is a humanuser that carries the GPS device.
 4. The method of claim 3, wherein theGPS device is included in a head-mounted device worn by the human user.5. The method of claim 1, wherein automatically correlating the aerialphotograph of the search region to the geo-tagged photograph of thesearch region includes communicating the aerial photograph to a remoteserver, computing the automatic correlation at the remote server, andreturning the computed automatic correlation to an assistive deviceassociated with the GPS device.
 6. The method of claim 1, whereinautomatically correlating the aerial photograph of the search region tothe geo-tagged photograph of the search region includes communicatingthe aerial photograph to an assistive device and computing the automaticcorrelation at the assistive device.
 7. The method of claim 6, whereinthe assistive device is configured to download a regional image databaseincluding the geo-tagged photograph and pre-defined geospatialcoordinates.
 8. The method of claim 1, wherein the aerial photograph ofthe search region is captured by a camera on a flying human-portabledevice deployed by a human user.
 9. The method of claim 1, wherein theaerial photograph of the search region is captured by a camera on anautonomous flying vehicle configured to automatically deploy to anaerial region substantially above the visual target for capturing theaerial photograph.
 10. The method of claim 1, wherein the aerialphotograph of the search region is captured by a camera on a flyingvehicle, and wherein the flying vehicle is configured to capturenear-infrared photographs and wherein the aerial photograph and thegeo-tagged photograph include near-infrared features.
 11. The method ofclaim 1, wherein automatically correlating the aerial photograph of thesearch region to the geo-tagged photograph of the search region includesautomatically computer-assessing a homography between the aerialphotograph and the geo-tagged photograph.
 12. The method of claim 11,wherein computer-assessing the homography is based on comparing a set ofderived image features of the aerial photograph having a smallerdimensionality than the aerial photograph, to a corresponding set ofderived image features of the geo-tagged photograph having a smallerdimensionality than the geo-tagged photograph.
 13. A method ofautomatically geolocating a visual target, comprising: affirmativelyidentifying a visual target in an aerial photograph of a search regioncaptured by a flying vehicle operating in the search region, the visualtarget having an assistive device including a global positioning system(GPS) device; automatically correlating the aerial photograph of thesearch region to a geo-tagged photograph of the search region, whereinthe geo-tagged photograph is labelled with pre-defined geospatialcoordinates; based on such automatic correlation, determining ageospatial coordinate for the visual target in the search region;receiving from the assistive device a measured location assessed by theGPS device; detecting an error in the measured location assessed by theGPS device based on a disparity between the determined geospatialcoordinate and the measured location assessed by the GPS device; andassessing a corrective model for the measured location assessed by theGPS device based on the disparity.
 14. The method of claim 13, furthercomprising: applying the corrective model to the measured locationassessed by the GPS device to generate a corrected measured locationthat corrects the detected error.
 15. The method of claim 14, furthercomprising: sending the corrected measured location to the assistivedevice.
 16. The method of claim 13, wherein the corrective model is acorrective clock model that accounts for clock skew in compromised GPSsignals of the GPS device that cause the error in the measured locationassessed by the GPS device.
 17. The method of claim 13, wherein thecorrective model is a machine learning model configured to correct thedetected error based on a learned correlation between the determinedgeospatial coordinate and the measured location assessed by the GPSdevice.
 18. A method of automatically geolocating a visual target,comprising: affirmatively identifying a visual target in an aerialphotograph of a search region captured by a flying vehicle operating inthe search region, the visual target having an assistive deviceincluding a global positioning system (GPS) device; automaticallycorrelating the aerial photograph of the search region to a geo-taggedphotograph of the search region, wherein the geo-tagged photograph islabelled with pre-defined geospatial coordinates; based on suchautomatic correlation, determining a geospatial coordinate for thevisual target in the search region; receiving from the assistive devicea measured location assessed by the GPS device; detecting an error inthe measured location assessed by the GPS device based on a disparitybetween the determined geospatial coordinate and the measured locationassessed by the GPS device; assessing a corrective model for themeasured location assessed by the GPS device based on the disparity; andapplying the corrective model to the measured location assessed by theGPS device to generate a corrected measured location that corrects thedetected error.